Role of Emoticons in Sentence-Level Sentiment Classification

Automated sentiment extraction from social media is enabling technology to support gathering online customer insights. The basic sentiment extraction is semantic classification of a text unit as positive or negative using lexical and/or contextual clues in a natural language system. From the input side, it is observed that social media as a sub-language often uses emoticons mixed with text to show emotions. Most emoticons, e.g. :=), are not natural language words, but textual symbols using characters to present a smiley face. Intuitively, such symbols are innately associated with emotions, whether happy, annoyed or don’t care, hence important clues for helping sentiment classification. Previous research has involved the limited use of emoticons as noisy labels in sentiment learning but detailed study on how noisy or useful they are has not been done. This paper presents a comprehensive data analysis study of the role of emoticons in sentence level sentiment classification. Various investigations are conducted on a fairly large annotated social media corpus, selected by our consumer insight analytics system. This corpus consists of 40,548 sentiment-rich sentences which business users are truly interested in mining. The study shows that the consistency between positive/negative emoticons with human judgment in this corpus is as high as 75.2%. Another larger randomly selected corpus consisting of 300,000 sentences from social media shows its consistency with human judgment to be 40.1%. A further study finds that emoticons’ recall contribution to sentiment classification is moderate, nevertheless, the data containing emoticons and brands are guaranteed to be quality social media representing customers’ voice instead of businesses’ voice such as press news. In addition, emoticon is an additional factor to help extract sentiments where other linguistic clues are insufficient.